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ABSTRACT 

Although college counseling agencies serve many 
clients, published accounts of counseling outcome;!^ ^n these agencies 
are relatively rare. An important aspect of counseling evaluation is 
the client's resolution of presenting and subsequent problems. This 
stud/ investigated the impact of counseling on university students 
us in J a measure of symptom relief, obtained by comparing the client *s 
own rated severity of presenting problem at intake with ^ 
posttreatment rating, and a measure of client satisfaction. The 
sample consisted of 102 female and 53 male clients and a control 
group of 47 females and 8 males. Chi-square analysis revealed a 
higher return for controls (84%) than fot clisfxts (54%) (p<.01). 
T--tests performed on the target-complaint data showed that female 
clients reported more improvement on their presenting problem than 
did male clients (p<.01) and that female clients reported more 
improvement than did female controls (p<.05). Inconsistency of 
individual client responses on the two outcome measures is 
interpreted in terms of the support function provided by the 
counselor. The importance of using both measures in future research 
is stressed. (Author/NB) 
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Abstract 

The present study investigates the in^)act of counselling on 
university students using a measure of symptom relief 
(obtained by con5)aring the client's own rated severity of 
his/her presenting problem at intake with a posttreatment 
rating) and a measure of client satisfaction. The sample 
consisted of 102 female and 53 male clients and a control 
group of 47 females and eight males. Chi-square analysis 
revealed a higher return for controls (84%) than for clients 
(54%) (p<.01). T-tests performed on the target-complaint 
data showed: 1) female clients reported more improvement on 
their presenting problem than male clients (fi<.01) and 2) 
female clients reported more improvement than female 
controls (b<.05). Inconsistency of individual client 
responses on the two outcome measures is interpreted in 
terms of the support function provided by the counsellor. 
The importance of using both measures in future research is 
stressed. 



Counselling evaluation t 

3 



Although college counselling agencies serve huge numbers 
of clients with seriously debilitating problems, published 
accounts of counselling outcomes in these agencies are 
relatively rare (Farnsworth, 1966; Rosen & Zytowski, 1977). 
The research which does exist has been restricted in nature. 
Some researchers have developed descriptive blueprints for 
evaluation (Lewis & Hutson, 1983; Mines, Grissard & Daniels, 
1982; Getting, 1982), while others have focused on the 
efficiency of the counselling process rather than on the 
outcome (Troy & Magoon, 1979). Still others have evaluated 
very specific forms of counselling intervention utilized 
with a strictly defined clientele (Gulanick, Howard & 
Moreland, 1979; Nagelberg, Hale & Ware, 1984; Wankowski, 
1979). 

Research examining the effectiveness of counselling 
services from the client's perspective have most frequently 
employed retrospective self-reports of client satisfaction 
with services. Various facets of this outcome measure have 
been explored by Greenfield (1983), Miller (1978), and 
Murillo, Shaffer and Michael (1981). Satisfaction ratings 
are relatively easy to demonstrate compared to other 
outcome measures and this has "given rise to professional 
suspicion about the use of these measures" (Greenfield, 
1983, p. 316). It is likely that general client 
satisfaction measures are affected by factors other than by 
counselling effectiveness. 
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An important aspect of counselling evaluation is the 
client's resolution of her/his presenting and subsequent 
problems. However, measures of change in terms of symptom 
relief or goal attainment that could be attributed to the 
counselling Intervention are not common. Weinstein and 
Ricks (1977) and Paritzky and Magoon (1982) have described 
applications of the Kiresuk and Sherman (1968) goal 
attainment scaling approach to counselling. And, the 
target-complaint technique (Battle, Imber, Hoehn-Saric, 
Stone, Nash & Frank, 1966) which provides an index of 
problem reli^ef has been utilized by Rosen and Zytowski 
(1977) in a university counselling setting. 

The current study was initiated by a search Eor an 
outcome measure of overall effectiveness of a university 
counselling agency. The university counselling setting is 
unique in several respects. First, although full-time 
counsellors are employed, the majority of counsellors are 
interns in the clinical psychology program. Therapeutic 
orientations vary considerably. The interns usually stay 
with the Center for only one year, and programs are 
developed in response to student need/demand. This moans 
that change is ongoing. Thus, an evaluative measure ia 
required which can address the effectiveness of the 
counselling agency as a whole. The measure must be 
sufficiently general to permit application across various 
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programmes with different formats and goals, across various 
counsellors of different therapy orientations and across 
clients with different presenting issues; yet, sufficiently 
customized so that the idiosyncratic features of each 
client's presenting issues are taken into account. Further, 
,it is essential that the measure be relatively unobtrusive 
in terms of infringement on the client, counsellor and 
counselling process. Finally, feasibility of implementation 
in terms of a minimal resource commitment is an important 
concern. 

The target-complaint technique is a customized evaluative 
instrument which allows for clients' idiosyncratic problem 
definitions. It appeared to be the measure which best fit 
with the criteria outlined above. Rosen and Zytowski 
(1977), using this technique, studied a large group of 
university counselling services clients. Th^^ present study 
is an attempt to replicate and extend the work of Rosen and 
Zytowski (1977) with th^s measure. 

The target-complaint technique allows a measure or relief 
to be obtained by comparing the client's uwn rated severity 
of his/her presenting problem at intake with a rating 
obtained after treatment. Although the primary measure of 
counselling effectiveness used in this study was the degree 
of relief rating, the follow-up questionnaire (modelled 
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closely after that used by Rosen and Zytow?kl, 1977, (Rosen, 
1985)) also Included items concerning the emergence of new 
problems and their resolution, the client *s perceived need 
to resume counselling, whether indeed additional counselling 
had been obtained, and the professional orientation of the 
"new" counsellor. Also included were several questions 
related to the degree of change in specific areas (i.e., 
relating to others; hopefulness about the future). The 
present study also extends the preliminary work of Rosen and 
Zytowski (1977) by analysing the data by client sex and 
incorporating a non-counselled control group. Further, in 
an attenpt to determine the relationship between degree of 
relief anJ client satisfaction, the latter was assessed 
using a modified form of the most commonly utilized measure, 
the Counseling Services Assessment Blank (Hurst, Weigel, 
Thatcher & Nyman, 1969). 

Method 

Subjects 

The original client sample consisted of 155 student clients 
(102 females and 53 males) who requested counselling at the 
Counselling and Development Centre of York University during 
the 1985/86 academic year, and who agreed to participate in 
this study. Their ages ranged from 17 to 57 years with mean 
ages of 24.6 years and 25.7 years respectively. Over two- 
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thirds of the clients were full-time students in the Faculty 
of Arts* Since it is the policy of the Centre to provide 
counselling on request, it was not possible to obtain a 
control group of "wait-listsd" clients. Instead, an initial 
control group of 65 students (56 females and nine males), 
was recruited from undergraduate psychology classes. Their 
ages ranged from 20 to 36 years of age, with mean ages of 
22,2 years and 23,1 years respectively. The majority of 
these subjects were full-time students in the Faculty of 
Arts. These respondents volunteered to take part in the 
research which was described as an investigation of the 
problems and stresses of students* Nine control subjects 
(one male and eight females) were excluded from the analyses 
due to their participation in therapy at some point during 
the data collection period. One additional female control 
subject did not provide useable pretest data. 

Procedure 

Clients in the sample were given a packet of questionnaires 
prior to, or if this was not feasible^ immediately 
following their first counselling session. Included in this 
package was a brief demographic background questionnaire and 
the target-complaint form, which required an identification 
by the client of the problem he/she wished help with. 
Clients rated the severity of this problem on a five-point 
scale. A similar form was included in the materials 
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administered to respondents in the control condition with 
minor rewording deleting the specific reference to 
counselling involvement. 

In May^, all respondents were mailed a posttest packet 
which included the target-coniplaint form. The oria\.al 
problem as stated by the subject appeared on the form along 
with a request to rate the severity of the problem at the 
present time. Additional items required ratings of feelings 
about the self and relationships, effectiveness of 
activities and confidence in the future, as well as an 
exploration of the emergence of new problems and whether the 
respondent (if a client} sought further counselling. Also 
enclosed in the packet to clients was a shortened version of 
the Counselling Services Assessment Blank. The entire 
package was sent out again one month later to those 
participants who failed to return the first set of follow-up 
questionnaires. 

All interested participants were given the opportunity to 
discuss their questionnaires with a research assistant at 
the end of the study. 
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Results and Discussion 

Of the 210 questionnaire packets mailed out, 130 were 
returned a return rate of 62%. As in the Rosen and 
Zytowski study, many of the clients (n=10) did not receive 
th55 packets because they had moved and left no forwarding 
address. The return rate from respondents in the control 
group (84%) was significantly better than for clients (54%), 

Ml/ N = 127) = 14.92, B<.01. A comparison of the 
return rates for female clients (57%) versus female controls 
(85%) similarly revealed a significant difference between 
these two groups ^ (1, N = 95) = 11.44, b<.01. Male and 
female client groups did not differ significantly from the 
57% rate obtained by Rosen and Zytowski (1977) especially 
when the undelivered packets are taken into account in which 
case the current study *s rate of return was 58%. Kale and 
female respondents in the control group had a return rate of 
75% and 85% respectively. Since the pmall sas^le size for 
control males (posttest n=6) limits confidence in these 
results, data from this group are not reported separately. 

Two types of data are available from responses to the 
target complaint form. The first type involves the 
qualitative data derived from the subject's written 
description of her/his presenting problem (or target 
complaint). As might be expected in a sample of university 
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students, the raaiority of clients and controls reported some 
form of academic concern as their target complaint, most 
typically, problems related to time management. There were 
no statistically significant differences between male (35%) 
and female clients (44%) or female clients and female 
controls (45%) in the frequency of academic target 
con^^laints. Also commonly reported by all groups were 
relationship problems and again, there were no significant 
differences between male (27%) and female (40%) clients and 
female clients and controls (25%). However, a closer look 
at the nature of the relationship difficulties revealed that 
significantly more female clients than either male clients, 
( Ml/ N = 81) = 5.46, E<.05), or female controls ( » 
(1, N = 95) = 6*92, E<.01), reported concerns regarding 
relationships with relatives, especially parents. 
Unpleasant feelings (i.e., feeling stressed or pressured; 
low self-esteem; low self-confidence) were frequently 
mentioned as target complaints. There were no differences 
between male (31%) and female (42%) client frequencies but 
significantly fewer female controls (22%) mentioned negative 
affects as compared with feriiale clients, ^ (1, N = 95) = 
3.87, E<«05. One area reported by significantly more female 
controls (15%) than female clients (4%) involved career 
related concerns, ^ (1, N = 95) = 3.88, e<*05. 
Frequencies of other problem types (sexuality concerns; 
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financial; weight related) were too low to pemit 
statistical comparisons • It should be noted that 
respondents often mentioned concerns which combined one or 
more of the above problem areas, hance. the combined 
percentages exceed 100. 

The second type of data derived from the target complaint 
form is the severity rating of the complaint. For each 
subject who returned the posttest questionnaire, a pretest 
and posttest severity rating were analysed. The other datum 
analysed was the 'degree of relief* score. This was 
calculated simply by subtracting the posttest severity 
rating from the pretest severity rating. Positive scores 
indicate reduction in tae perceived severity of the 
presenting problem; negative scores indicate an increase in 
the severity of the problem. 

To ascertain whether those subjects who returned the 
posttest questionnaires were significantly different from 
non-responding subjects in a systematic way, t-tests were 
performed comparing the pretest severity scores collapsing 
across sex of subject. Pretest severity scores did not 
differ within the client group or control group as a 
function of whether posttest data was obtain»id (client: no 
posttest M=3,89, SD=,85, posttest M=3,88, SD=,84 coL^rolt 
no ponttest K=3,33, SD=,50, posttest M=3,33, SD=,84), 
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Table 1 presents the mean initial and subsequent rating 
of the target complaint for eanh subject group. Higher 
scores reflect a greater degree of concern, worry or 
discomfort vis a vis the problem. Also presented are the 
means and standard deviations for the degree of relief 
scores 0 Although 58 female clients returned the posttest 
questionnaire package, three failed to con^lete the target- 
coraplaint form on the basis that they had received only one 
counselling session. Thus, the n for treatment females in 
Table 1 is 55. 



Insert Table 1 about here 



T-tests were performed con5)aring the pretest severity 
ratings, posttest severity ratings, and degree of relief 
acores across the control and client groups. As shown in 
Vable 1, these tests revealed that: 1) female clients rated 
their target complaint as more severe on pretest than did 
female controls (t(83.5) = -3.93, fi<.01), 2) female clients 
changed more over the period of time between pre- and 
posttesting than female controls (as measured by degree of 
relief) (t(68.3) = -2.10, b<.05) resulting in posttest 
severity ratings which did not differ from female controls, 
3) female clients rated their target complaints as wore 
severe on pretest than male clients (t(53) = 2.77, £<.01) 
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and, 4) female clients experienced more relief with respect 
to their target complaint than male clients by posttest 
(t(48.1) = 2.62, B<.01) resulting in equivalent posttest 
ratings. Direct comparison of the degree of relief scores 
with the Rosen and Zytowski (1977) report is not possible 
since they utilized a 9-point scale, however it appears that 
the average index of relief for the treatment group in the 
present study is comparable {M=1.15, SD=1.12) to the Rosen 
and Zytowski mean score converted into a five-point rating 
(M=1.30). 

Although all groups on average, reflect some improvement 
or relief with respect to the target complaint at the time 
of posttesting, not all change was positive. The frequency 
distribution in Table 2 for each magnitude of relief for 
each subject group, reveals that five percent of clients 
reported deterioration while 25% noted no change in their 
degree of concern for tneir problem. Thus, although the 
majority of clients reported some degree of relief vis a vis 
their target complaint, 30% reported no change or 
deterioration effects. Rosen and Zycowski (1977) report 
that 23% of their clients felt no inprovement or a 
worsening of their initial presenting problem. Since Rosen 
and Zytowski (1977) did not include a control group in their 
study, they could not compare these rates with the rate of 
worsening of problems found in the general population of 
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students. In the present study, this was found to be very 
revealing as 41% of the control subjects showed no 
inproveraent or worsening of their target complaints. This 
rate is considerably higher than that for the client sample, 
and clearly shows the importance of control groups in 
evaluation studies of this type* 



Insert Table 2 about here 



A chi-square analysis comparing control and client groups in 
terms of the proportion of subjects reflecting improvement 
versus no change versus deterioration revealed a significant 
difference between the two groups, ^ (2, N = 127) = 6.84, 
£<.05» Significantly more clients than controls experienced 
relief of their target conplaint. When female clients and 
controls were compared, more female clients were shown to 
have experienced improvement than female controls, ^ (2, N 
= 95) = 9*17, B<.01. Female clients were also compared with 
male clients in this same manner • Significantly more female 
clients reported improvement than male clients, M2, N = 
81) = 6.39, £<.05. 

It is ten^ting to conclude, at least in the case of 
female clients, that participation in counselling produced 
beneficial effects in terms of alleviating the degree to 
which a presenting problem was bothersome. However, the 
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possible operation of a regression effect must be 
considered. Since female clients had the highest pretest 
scores (H=4.05), and males and females differed 
significantly on pretest, there is greater opportunity among 
female clients for a shift in rating on posttest in the 
direction of relief. A chi-square analysis based on the 
extremity of pretest scores of male and female clients 
failed to reach an acceptable significance level. This 
analysis showed that extreme ratings were equally frequent 
for males and females, thus supporting an argument that the 
sex difference in the client group is not solely an artifact 
of regression to the mean. This finding of the greater 
efficacy of counsellinc} for female clients is intriguing and 
may be related to the different sex and gender role 
orientations of males and females. Prescriptions on males 
to repress emotionality and "to keep their selves to 
themselves" (Jourard, 1964, p. 47) may function to limit the 
effectiveness of at least some forms of counselling 
interventions. 

More than two-thirds of both the clients and non-client 
respondents reported the appearance of additional problems 
~ a considerably higher proportion than the 40% found by 
Rosen and Zytowski (1977) for their client sample. However, 
the mean reported resolution of this "new" problem for the 
treatment group (on a scale of -1 to 3) was 1.70 — slightly 
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higher than the mean for the Rosen and Zytowski clients 
(1.53). There was a trend toward female clients having 
higher resolution scores (1.79) than both male clients 
(1.43) and female controls (1.55). These differences were 
not significantly different but the trends are in accord 
with the greater efficacy of counselling for females. More 
than three-quarters of the client group reported a felt need 
for further counselling (50% in the Rosen and Zytowski 
sample) but only 30% actually did seek additional 
counselling. This result is con^)arable to that obtained by 
Rosen and Zytowski. Of those who did seek further help, 
relatively few (38%) consulted professionals — again, a 
finding similar to that reported in the earlier study. 
Those clients (n=13) with no felt need to resume counselling 
had significantly higher relief scores (t(17.5) = -3.32, 
fi<.01). This finding adds validity to the target-complaint 
measure as an appropriate technique for assessing the 
effectiveness of counselling. 

Four items on the follow-up questionnaire concerned the 
extent to which respondents perceived deterioration, no 
change or improvement (in the interval between the two 
testings) in four areas { feelings about the self; 
relationships with others; effective performance of 
activities; feelings about the future. Chi-square analyses 
con?)aring male versus female clients and female clients with 
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female controls on each of the four items revealed only one 
significant difference. Significantly more female clients 
than female controls felt more confident about the future, 
2 (1, N = 92) = 5.66, E<.02, at the time of posttesting as 
con^ared with the time of initial testing. 

The analysis of the Counselling Services Assessment Blank 
was complicated by several factors. The original authors 
(Hurst, Weigel, Thatcher & Nyman, 1969) analysed each item 
on the CSAB separately which increases the error rate to an 
unsatisfactory level. A total CSAB score obtained by 
summing across all items would be influenced by the large 
number of missing values (as clients did not answer 
questions not applicable to them) therefore this would not 
be appropriate. Thus, it was decided to calculate the mean 
CSAB score for each client, based on those items to which 
she/he responded. Further analyses were performed when upon 
close examination the items in the CSAB did not all appear 
to be asking about client satisfaction; some asked about the 
clients' perception of benefit from services (questions 
1-6), some asked about issues of the Centre's reputation 
(questions 7,8 and 14), and some asked strictly about the 
client's satisfaction (questions 9-12,16). These clusters 
of questions were analysed separately to establish whether 
any of the clusters was a more accurate representation of 
the success of counselling services. Group and individual 
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counselling clients were collapsed following a 
nonsignificant con?)arison on CSAB scores* The only 
significant difference between these two groups was that the 
group clients were more positive about the physical 
environment and reputation of the Centre, t(ll,3)=2,83, 
Ei<»05* 

All clients generally perceived their counselling 
involvement positively. Female clients scored significantly 
higher on the CSAB, indicating greater satisfaction than 
male clients, t(48,4) = 2,08, fi<,05. When these CSAB scores 
were broken down into the question clusters, no sex 
differences were found on the satisfaction questions, or on 
the reputation of the Centre, Sex differences were evident 
in terms of the perceived benefits of the counselling 
however, with female clients perceiving more benefit from 
the counselling experience, t ( 53,2) = 2,14, 05, A 
correlation performed between mean CSAB scores and degree of 
relief scores suggests that the two measures are related (r 
= .32), however considerable individual variability was 
evident. Both the satisfaction and benefit clusters were 
correlated with the degree of relief scores however to a 
lesser extent. Not surprisingly, the questions pertaining 
to the reputation of the Centre were unrelated to degree of 
relief scores, These correlations, while significant, are 
not indicative of a massive overlap between the two 
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measures. Twenty-five percent of those who regarded their 
overall counselling experience very positively reported no 
change or deterioration on the target complaint while 57% of 
those who viewed the counselling experience as neutral or 
negative did in fact report some degree of relief on the 
target complaint. 

Also difficult to explain is the lack of congruence 
between ratings of the other potential benefits of 
counselling (in areas of the enhancement of self-knowledge, 
improvement in feelings about the self, acquisition of 
needed information, resolution of internal conflict, 
resolution of interpersonal conflict, improvement of a 
skill, as measured by individual items on the CSAB) and 
ratings of the overall counselling experience. Average 
ratings on such items viire typically in the neutral range 
(i.e., counselling was only of low to moderate benefit) yet 
the overall counselling experience was quite positively 
rated by the majority of clients. 

For many clients it appears that a positive view of the 
counselling experience may be relatively independent of 
resolution of the presenting problem and/or self-growth and 
development. One possible construction of these findings is 
that for some clients, although counselling does little to 
solve their initial problem and does not lead to 
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Irnproveraents in other areas, it may enhance the clients' 
ability to tolerate the discomfort occasioned by their life 
stresses and hence is viewed by these consumers as a 
positive, helpful experience* Indirect support for this 
speculation derives from the research of Goodman, Sewell and 
Jampol (1984) and Bosmajian and Mattson (1980) ♦ These 
investigators reported that counselling seekers have less 
access to (or make less use of) alternate sources of help 
and support (i.e., family and friends). The data obtained 
from the target-complaint form suggests that this may be 
reinforced by the presence of specific problems existing in 
the family and peer relations, especially for female 
clients. Thus, perhaps the counselling experience is 
perceived as beneficial, in the absence of significant 
problem amelioration or self -growth, because of the support 
function provided by the counsellor. More detailed 
exploration of this hypothesis must await further research. 

The generalizability of this study is restricted by the 
small control group, particularly the absence of male 
controls. However this is mediated by the valuable 
Information gained by the inclusion of a control group. In 
the future, evaluation studies should consider matching 
control subjects with clients on demographic characteristics 
in order to gain the Post insight into the benefits of the 
counselling services. The use of the target-complaint 
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technique in the assessniRnt of this counselling centre was 
found to be extremely valuable, giving information not 
possible through measures of client satisfaction alone. The 
two types of measures, in combination, provided data with 
greater richness and complexity than usually gathered in 
evaluation stud.-l^s of university counselling services and 
are recommended for future research. 

Footnotes 

1. Any client in an obviously severely stressed condition 
was not asked to participate in the research prior to 
their intake session. 

2. The timing of the posttest mailing was arranged so that 
no client would receive i\ follow-up packet less than one 
month after their first therapy session. 
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Table 1 

Mean Ratings for the Target Complaint 



Subject Group 


Pretest 


Posttest 




Degree of 










Relief 




n M 


M 


M 


SD 


Treatment females 


55 4.05 


2.69 


1.36 


1.06 


Treatment males 


26 3.54 


2.85 


0.69 


1.09 


Control females 


40 3.38 


2.58 


0.80 


1.27 
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Table 2 

Frequency Distribution of Relief Scores 



Degree of Treatment Treatment Control 
Relief Females Males Females 





n 


i 


n 


% 


n 


* 


-3 


0 


0 


0 


0 


1 


1 


-1 


1 


2 


3 


12 


7 


18 


No change 


11 


20 


9 


35 


8 


20 


+1 


20 


36 


9 


35 


12 


30 


+2 


13 


24 


3 


12 


6 


15 


+3 


10 


18 


2 


8 


6 


lb 
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